Skip to content

Support all negative values in ClassLabel #4511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jun 16, 2022
Merged

Support all negative values in ClassLabel #4511

merged 2 commits into from
Jun 16, 2022

Conversation

@lhoestq
Copy link
Member

@lhoestq lhoestq commented Jun 16, 2022

We usually use -1 to represent a missing label, but we should also support any negative values (some users use -100 for example). This is a regression from datasets 2.3

Fix #4508

@lhoestq lhoestq requested a review from mariosasko June 16, 2022 09:59
lhoestq
lhoestq commented Jun 16, 2022
View reviewed changes
@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Jun 16, 2022 •
edited
Loading

The documentation is not available anymore as the PR was closed or merged.

mariosasko
mariosasko approved these changes Jun 16, 2022
View reviewed changes
Copy link
Collaborator

@mariosasko mariosasko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM!

@lhoestq lhoestq merged commit dced38b into master Jun 16, 2022
@lhoestq lhoestq deleted the support-all-negative-values-in-classlabel branch June 16, 2022 13:54
khushmeeet pushed a commit to khushmeeet/datasets that referenced this pull request Jun 23, 2022
@lhoestq @khushmeeet
Support all negative values in ClassLabel (huggingface#4511)
b335481 
* support all negative valeus in ClassLabel

* support None in cast_storage
@davidbenton
Copy link

davidbenton commented Jun 23, 2022 •
edited
Loading

Thanks for this fix! I'm not sure what the release timeline is, but FYI #4508 is a breaking issue for transformer token classification using Trainer and PyTorch. PyTorch defaults to -100 as the ignored label for negative log loss, so switching labels to -1 leads to index errors using Trainer defaults.

As a workaround, I'm using master branch directly (pip install git+https://github.com/huggingface/datasets.git@master for anyone who needs to do the same) until this gets released.

@lhoestq
Copy link
Member Author

lhoestq commented Jul 28, 2022

The new release 2.4 fixes the issue, feel free to update datasets :)

pip install -U datasets

@l-uuz
Copy link

l-uuz commented Jul 23, 2025

@lhoestq I hope it's OK to ping you here. I've noticed that encode_example does only work with -1. I already created #7645 to fix the documentation, but then I stumbled across your original changes to the docs text in this PR.

I am talking about this part in ClassLabel -> encode_example:

datasets/src/datasets/features/features.py

Lines 1126 to 1129 in e71b0b1

# Allowing -1 to mean no label.
if not -1 <= example_data < self.num_classes:
raise ValueError(f"Class label {example_data:d} greater than configured num_classes {self.num_classes}")
return example_data

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mariosasko mariosasko mariosasko approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

cast_storage method from datasets.features

6 participants

@lhoestq @HuggingFaceDocBuilderDev @davidbenton @l-uuz @mariosasko